Sparse Is Enough In Scaling Transformers | Ml Research Paper Explained

Sparse is Enough in Scaling Transformers (aka Terraformer) | ML Research Paper Explained

Soft Mixture of Experts - An Efficient Sparse Transformer

AI Papers Academy

Sparse LLMs at inference: 6x faster transformers! | DEJAVU paper explained

AI Coffee Break with Letitia

Transformers: The best idea in AI | Andrej Karpathy and Lex Fridman

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Scaling Transformer to 1M tokens and beyond with RMT (Paper Explained)

Sparse Transformers and MuseNet | AISC

LLMs Explained - Aggregate Intellect - AI.SCIENCE

Giannis Daras: Improving sparse transformer models for efficient self-attention (spaCy IRL 2019)

Attention mechanism: Overview

Google Cloud Tech

Big Bird: Transformers for Longer Sequences (Paper Explained)

LongNet: Scaling Transformers to 1B tokens (paper explained)

Sparse Expert Models (Switch Transformers, GLAM, and more... w/ the Authors)

LongNet: Scaling Transformers to 1,000,000,000 Tokens Explained

Gabriel Mongaras

Barret Zoph Switch Transformers: Scaling to Trillion Parameter Models w/ Simple & Efficient Sparsity

Pretrained Transformers as Universal Computation Engines (Machine Learning Research Paper Explained)

CVPR2023 Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers

Transformers, explained: Understand the model behind GPT, BERT, and T5

Google Cloud Tech

Transformer Neural Networks, ChatGPT's foundation, Clearly Explained!!!

StatQuest with Josh Starmer

BigBird Research Ep. 1 - Sparse Attention Basics

ChrisMcCormickAI